SHELLING THE COSET POSET 



RUSS WOODROOFE 



Abstract. It is shown that the coset lattice of a finite group has shellable 
order complex if and only if the group is complemented. Furthermore, the 
coset lattice is shown to have a Cohen-Macaulay order complex in exactly the 
same conditions. The group theoretical tools used are relatively elementary, 
and avoid the classification of finite simple groups and of minimal finite simple 
groups. 



1. Introduction 

We start by recalling the definition of a shelling. All posets, lattices, simplicial 
complexes, and groups in this paper are finite. 



Definition 1.1. If A is a simplicial complex, then a shelling of A is an ordering 
.Ft, . . . , F n of the facets (maximal faces) of A such that Fk n (u^" 1 -?^) is a nonempty 
union of facets of Fk for k > 2. If A has a shelling, we say it is shellable. 



We will use this definition in the context of a poset P by recalling the order 
complex \P\ to be the simplicial complex with vertex set P and faces chains in P. 
We say that a poset P is shellable if \P\ is. Recall also that P is graded, and \P\ is 
pure, if all maximal chains in P have the same length. 

The idea of a shelling (and the property of shellability) were first formally in- 
troduced by Bruggesser and Mani in [Hj, though similar ideas had been assumed 
implicitly since the beginning of the 20th century. See Chapter 8 of j2j for a 
development of some of the history and basic results on shellability. Since its intro- 
duction, it has been studied extensively by combinatorialists. Particularly, in the 
1980's and 90's Bjorner and Wachs wrote several papers QU^EU developing the 
theory of shellability for posets. Especially important to those interested in group 
theory are jH] and (Sj, as they extend the older definition of shellability (which only 
applied to graded posets) to apply to any poset. This extension makes Theorem 
11.31 much more interesting! 

We henceforth assume that a reader has seen the basic definitions and results of, 
say, although we try to state clearly what we are using. 

Recall that the subgroup lattice (denoted L(G)) is the lattice of all subgroups 
of a group G. Shellings of subgroup lattices have been of interest for quite some 
time. In fact, one of the main results of Bjorner's first paper on shelling posets j2j 
was to show that supersolvable groups have shellable subgroup lattices. (Recall a 
supersolvable group is a group having chief series with every factor of prime order.) 
As mentioned before, at that time, shellability was a property that applied only to 
graded posets. Under this definition, Bjorner had the shellable subgroup lattices 
completely characterized, if we recall the following theorem of Iwasawa: 
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Theorem 1.2. (Iwasawa 13J) Let G be a finite group. Then L(G) is graded if and 
only if G is supersolvable. 

Of course, when Bjorner and Wachs updated the definition of a shelling to allow 
non-graded posets in [HE], shellable subgroup lattices were no longer characterized. 
This gap was soon filled by Shareshian: 

Theorem 1.3. (Shareshian |19j l Let G be a finite group. Then the subgroup lattice 
L{G) is shellable if and only if G is solvable. 

A nice summary article on shellability and group theory was written by Welker 
in [2Bj- This article is now somewhat out of date, and it has some errors, but it 
is very useful as an overview of the topic. The reader should be warned, however, 
that at the time it was written shellability was still considered to apply only to 
graded posets. 

Shareshian's result is surprising and pretty, and it would be nice to find some- 
thing similar for other lattices on groups. In this paper, we consider cosets. The 
coset poset £(G) (poetically named by K. Brown in [Jj) is the poset of all cosets 
of proper subgroups of G, ordered by inclusion. The coset lattice £(G) is £(G) U 
{G, 0}, likewise ordered by inclusion. The meet operation is intersection, while 
H\Xi V H 2 X2 = (Hi, H 2 , x\X2 1 )x2- Clearly, £(G) is shellable if and only if £(G) 
is, so we study the two interchangeably. If £(G) is shellable, we will call G coset- 
shellable. 

The history of the coset poset is discussed in the last chapter of JHj- Most results 
proved have been either negative results, or else so similar to the situation in the 
subgroup lattice as to be uninteresting. More recently, K. Brown rediscovered the 
coset poset, and studied its homotopy type while proving some divisibility results 
on the so-called probabilistic zeta function In particular, he showed that if G 
is a solvable group, then |£(G)| has the homotopy type of a bouquet of spheres, all 
of the same dimension. 

In Section we show that there are finite groups G which have a shellable 
subgroup lattice, but a non-shellable coset lattice. In particular, we show that 
for £(G) to be shellable, G must be supersolvable, and every Sylow subgroup of 
G must be elementary abelian. Our main tool is the above-mentioned result of 
Brown, together with the fact that a pure shellable complex is homotopic to a 
bouquet of top-dimensional spheres. In Sectional we use linear algebra to construct 
an invariant on subgroups of such groups. Finally, in SectionQ]we use this invariant 
to construct a so-called EL-shelling, and to finish the proof of our main theorem: 

Theorem 1.4. (Main Theorem) Lf G is a finite group, then €(G) is shellable if 
and only if G is supersolvable with all Sylow subgroups elementary abelian. 

Our theorem is even more interesting when we connect it with a paper of P. Hall 
[12] . We recall that a group G is complemented if for every subgroup H C G, there 
is a complement K which satisfies i) K n H = 1 and ii) HK = KH = G. Hall 
proved the equivalence of the first three properties in the following restatement of 
our theorem: 

Theorem 1.5. (Restatement of Main Theorem) If G is a finite group, then the 
following are equivalent: 

(1) G is supersolvable with all Sylow subgroups elementary abelian, 
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(2) G is complemented, 

(3) G is a subgroup of the direct product of groups of square free order, 

(4) G is coset-shellable, 

(5) £(G) is homotopy Cohen- Macaulay, 

(6) £(G) is sequentially Cohen- Macaulay over some field, and 

(7) <£(G) is Cohen- Macaulay over some field. 

Parts (5), (6) and (7) are discussed in Section f2,31 where we define the three 
used versions of the Cohen-Macaulay property, and give further references. 

Notice that, contrary to the situation of Shareshian's Theorem, <£(G) is shellable 
if and only if it is pure and shellable. Thus, non-pure shellability only comes in in 
the negative direction of our proof. Of course, now that it has been defined, one 
can't ignore it! 

Complemented groups have also been called completely factorizable groups, and 
have been studied by other people, see for example pp, or [T($ . Ramras has further 
examined the homotopy type of the coset poset in [Tfi] . 

I would like to thank my advisor, Ken Brown, for many discussions and ideas; 
and Keith Dennis for helping with some of my group theory questions. Thanks 
also to Yoav Segev, Anders Bjorner, and the anonymous referees for their helpful 
comments. Though no computations appear in this paper, GAP [11 and the XGAP 
package often helped in searching for examples that furthered understanding. 

2. COSET POSETS THAT ARE NOT SHELLABLE 

2.1. p-groups. It is often easier to show that something is not shellable, than to 
show that it is. So we start our search for shellings of the coset lattice by finding 
groups for which £(G) is certainly not shellable. The following lemma will be very 
useful in this endeavor. 

Lemma 2.1. If P is a shellable poset, then every interval in P is also shellable. 
(Thus, if G is coset-shellable, then so is every subgroup H C G.) 

Proof. Since every interval in a poset P is a "link" (for more information, see the 
beginning of Section l2~3l) . the first part follows immediately from Proposition 10.14 
in 0. 

For the second part, we note that the interval [0, H] in <t(G) is isomorphic to 
<t(H), □ 

Corollary 2.2. If G is a finite coset-shellable group, then G is solvable. 

Proof. Note that the interval [1,G] in <£(G) is isomorphic to the subgroup lattice 
of G. Apply Lemma T2. II and Shareshian's Theorem (Theorem II .3(1 . □ 

A proof of Corollary 12 . 21 that does not rely on Shareshian's Theorem will also be 
given, in Section l2~2l 

At first glance, one might hope that perhaps all solvable groups have a shellable 
coset poset. Soon enough, however, one considers the coset-poset of Z4, pictured 
in figure ^ We see that £(^4) is not even connected, and connectivity is an easy 
consequence of the definition of shellability as long as all facets have dimension at 
least 1. 

A similar situation holds for an arbitrary prime p: Z p 2 has only one nontrivial 
proper subgroup, so £(Z p 2) falls into p connected components, and in particular 
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Figure 1. The coset poset of Z 4 is not connected, so not shellable. 

is not shellable. Hence, no group G with a subgroup isomorphic with Z p 2 can 
be coset-shellable. Can we eliminate any other p-groups from the possibility of 
coset-shellability? In fact we can. We recall the following theorem of K. Brown: 

Theorem 2.3. (Proposition 11 from 0) Let G be a finite solvable group with a 
chief series 1 = Nq C JVi C ■ ■ • C Nk — G. Then <£(G) has the homotopy type of a 
bouquet of (d — 1) -spheres, where d is the number of indices i = 1, . . . , k such that 
Ni/Ni-i has a complement in G/Ni-\. 

It follows from the proof in [Z] that for G ^ 1 the number of spheres is, in fact, 
nonzero. (It is 

k 

- dlNi/Ni-xl) 

i=l 

where a is the number of complements Ni/Ni-i has in G/Ni-i; also related is 
Corollary 3].) 

Proposition 2.4. 

1) If H is a finite p-group which is coset-shellable, then H is elementary abelian. 

2) If G is a finite group which is coset-shellable, then all Sylow subgroups of G are 
elementary abelian. 

Proof. 1) If H is a finite p-group, then L(H) is graded (by, for example, Iwasawa's 
Theorem, Theorem 1 1.2JI . hence €.(H) is also graded. But it is well known (see for 
example that a graded, shellable poset P has homotopy type of a bouquet of r- 
spheres, where r is the length of a maximal chain in P. By the above theorem, we see 
that every chief factor of H must be complemented, and hence H has trivial Frattini 
subgroup <&(-/?) (otherwise any minimal normal subgroup contained in $(-ff) is an 
uncomplemented chief factor) . 

But for a finite p-group, $(H) = H'H P (see for example [HI 5.3.2]), so H is 
abelian of exponent p, that is, elementary abelian. 

2) Apply Lemma, l2~D to the interval [0, H] in <t(G), where H is a Sylow subgroup 
ofG. □ 

2.2. Non-supersolvable groups. We now have that for a finite group G to be 
coset-shellable, G must be solvable with elementary abelian Sylow subgroups. A 
little more holds: G must in fact be supersolvable. To prove this, it suffices by 
Lemma l2~D and the discussion of the previous section to restrict ourselves to groups 
G such that: 

(1) G is not supersolvable, 

(2) All proper subgroups of G are supersolvable, and 

(3) All Sylow subgroups of G are elementary abelian. 
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A closely related idea is that of minimal non-complemented groups, which are 
non-complemented groups with every proper subgroup complemented. A complete 
characterization of such groups is given in > although we do not use their char- 
acterization. 

In light of Shareshian's Theorem and Corollarv l2.2l it might seem at first glance 
that a stronger version of Condition 1 would be to require G to be solvable but not 
supersolvable. The following result of Doerk, however, shows that this would be 
redundant. 

Lemma 2.5. (Doerk, |9l Hilfssatz C]) If every maximal subgroup of G is super- 
solvable, then G is solvable. 

We notice also that this frees our characterization of groups that are not coset- 
shellable from Shareshian's Theorem, which relies on Thompson's classification of 
minimal finite simple groups. We will also need this for Section l2~3l 

For any normal subgroup N in G, let q : G — > G/N be the quotient map. Then 
we take (£q(G) to be the subposet of £(G) of all Hx such that q{Hx) ^ G/N. Thus, 
Co(G) is obtained from C(G) by removing cosets Kx when KN — G. We will use 
the following proposition to show that, for G satisfying Conditions 1-3, <£(G) has 
the wrong homotopy type to be shellable. 

Theorem 2.6. (K. Brown 0, Proposition 8 and following discussion) The quotient 
map q : G — > G/N induces a homotopy equivalence €o(G) — * <L(G/N). 

The following lemma from group theory will be useful. 

Lemma 2.7. Let G be a solvable group, with H a proper subgroup. Then 

(1) If N is an abelian normal subgroup of G with NH = G, then H is maximal 
in G if and only if N/N n H is a chief factor for G. 

(2) H is maximal if and only if it is a complement to a chief factor Ni+x/Ni, 
i.e., if and only if HNi + i — G and H n Ni + i — iVj. 

Part 1 may be found in ^3 Theorem 5.4.2]. Part 2 follows from part 1 by taking 
Ni + i to be minimal such that HNi + \ = G. 
We use Lemma l2~TI in proving the following: 

Lemma 2.8. Let G be a group satisfying Conditions 1-3 above. Let n be the length 
of a longest chain in £(G). Then G has a minimal normal subgroup N , of non- 
prime order, over which \<Lq(G)\ is the subcomplex of |<£(G)| generated by all chains 
of length n. 

Proof. Our proof goes in five steps: 

1) Every chief factor Ni+i/Ni of G is complemented in G/iVj. 

We apply a theorem of Gaschiitz, proved in |14l Theorem 3.3.2], which says that 
a normal abelian p-subgroup N has a complement in G if and only if N has a 
complement in a Sylow p-subgroup P containing N. Let N i+ i/Ni be a chief factor 
of G. Then since G has all Sylow subgroups elementary abelian, G/Ni has all Sylow 
subgroups elementary abelian. But an elementary abelian group is a complemented 
group (see Theorem II .511 . hence Ni + \/Ni has a complement in any Sylow subgroup 
P/Ni containing it, and so by Gaschiitz we get that N+i/N has a complement in 
G/N. 

2) A chief factor Ni+i/Ni is of non-prime order only if Ni = 1. 
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Suppose otherwise, that N i+ i/Ni is of non-prime order with iV, 7^ 1, so that 
G/Ni is solvable but not supersolvable. Then Ni/Ni-i has a complement by part 
(1), so there is a group K with G/Ni = K/Ni-i. Since all subgroups of G are 
supersolvable, we see that G/Ni is supersolvable, a contradiction. 

3) There exists a minimal normal subgroup N C G of non-prime order, and TV 
is a complement to any maximal subgroup K of non-prime index. 

Since G is not supersolvable, there is some factor of non-prime order in any chief 
series of G, and by part (2) it must be of the form N\/l, We notice that in the 
situation of the second part of Lemma \2. 71 we have [G : H] = [iVj+i : Ni). Since 
Ni/1 is the only factor of non-prime order, we get the desired result. 

4) A maximal chain C has length less than n if and only if the top element of C 
is a coset Kx of some complement K of N. 

Suppose Hx is the top element of a chain C in C(G). By Iwasawa's Theorem and 
the supersolvability of H, we have that every maximal chain in the interval (0, Hx] 
has length n + 1 — a, where a is the number of primes with multiplicity dividing 
[G : H\. We see that C has length n if and only if a = 1, so C is of length less than 
n if and only if H is of non-prime index in G if and only if H is a complement of 
N. 

5) In the situation of part (4), C\ {Kx} can be extended to a chain of length n. 
Let K\x be the coset immediately under Kx in C. Then since K is supersolvable, 

[K : Ki] is a prime. Then [G : NK{\ — [K : K%] is also a prime. Moreover, NK\ 
is supersolvable, so if \N\ — p a , then there is a chain K\ = Hq < H\ < ■ ■ ■ < H a = 
NK\ between Kx and NK\. The desired chain then follows C up to ifii, and ends 
at the top with K\x < H\x < ■ ■ ■ < H a x — NK\x. 

We have shown that |Co(G)| is obtained from |C(G) by removing the facets of 
dimension less than n, thus that |£o(G) is the subcomplex of |<£(G)| generated by 
all n-faces. □ 

We notice in passing that the argument in part (2) actually shows that a com- 
plement K of N has CoreG K = 1, thus that a group G satisfying Conditions 1-3 
is primitive. Such groups have highly restricted structure, see Chapter A. 15 of QH] 
for an overview. 

We relate the preceding lemma to the following result from Bjorner and Wachs: 

Lemma 2.9. (Bjorner/ Wachs Theorem 2.9]) 7/ A is shellable, then the subcom- 
plex generated by all faces of dimensions between r and s is also shellable, for all 
r < s. 

We are now ready to prove our goal for this section. 

Theorem 2.10. If G is not supersolvable, then G is not coset- shellable. 

Proof. By the preceding discussion, it suffices to consider G solvable with every 
subgroup a complemented group. Let N be the minimal normal subgroup of order 
p a constructed in Lemma 12.81 Then the resulting |£o(G)| is the subcomplex of 
|£(G)| generated by the faces of dimension n. Theorem 12 . 61 gives us that |£o(G)| ~ 
\€{G/N)\. 

If £o(G) were shellable, then |£o(G) | would have the homotopy type of a bouquet 
of n-spheres, as discussed in the proof of Proposition 12.41 But Theorem 12.31 and 
the comment following give that |£o(G)| ~ \£(G/N)\ is homotopic to a non-empty 
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bouquet of (n — o)-spheres. Thus, £o(G) is not shellable, and by Lemma, l2i)l we see 
that <£(G) is not shellable. □ 

We have now proved that (4) => (1-3) in our Restatement of the Main 
Theorem, Theorem II .51 The following subsection, which the rest of the paper does 
not depend on, deals with (5-7). A reader who is unfamiliar with the Cohen- 
Macaulay property for simplicial complexes may, if desired, skip directly to Section 

m 

2.3. Cohen-Macaulay coset lattices. In fact, we have proven slightly more in 
Section I22J Two properties that are closely related to shellability are that of being 
Cohen-Macaulay and (generalized to non-pure complexes) that of being sequentially 
Cohen-Macaulay. Recall that the link of a face F in a simplicial complex A is 
lkA F = {F G A : F U F G A, F n F = 0}. Links in the order complexes of 
posets are closely related to intervals. More specifically, if C is a maximal chain 
containing x and y, and C is C with all z such that x < z < y removed, then it is 
easy to see that lkpC" is the order complex of the interval (x,y). In general, the 
link of a chain in a bounded poset is the so-called "join" of intervals. 

Let k be a field. A simplicial complex A is Cohen-Macaulay over k if for every 
face F G A, flj(lkA F, k) = for i < dimlkA F, i.e., if every link has the homology 
of a wedge of top dimensional spheres. It will come as little surprise after the pre- 
ceding discussion of links in posets that one can prove the following fact: a poset 
P is Cohen-Macaulay if and only if every interval (x, y) in P has the homological 
wedge of spheres property (see for a proof of this and further discussion of links 
and joins). The complex A is homotopy Cohen-Macaulay if every such link is ho- 
motopic to (rather than merely having the homology of) a wedge of top dimensional 
spheres. Since a graded shellable poset has the homotopy type of a wedge of top 
dimensional spheres, and since every interval in a shellable poset is shellable, we see 
that (the order complex of) a graded shellable poset is homotopy Cohen-Macaulay. 
A homotopy Cohen-Macaulay complex is Cohen-Macaulay over any field. 

There is an extension of the Cohen-Macaulay property to non-pure complexes. 
The pure i- skeleton of a simplicial complex A is the subcomplex generated by all 
faces of dimension i. We say that A is sequentially Cohen-Macaulay if its pure i- 
skeleton is Cohen-Macaulay for all i. A pure, sequentially Cohen-Macaulay complex 
is obviously Cohen-Macaulay. 

A reference for background on Cohen-Macaulay complexes is [213. Useful prop- 
erties of sequentially Cohen-Macaulay complexes are given in (22]. We recall some 
facts presented in the latter. 

Lemma 2.11. Let A be a simplicial complex, P a poset: 

(1) If A is shellable, then A is sequentially Cohen-Macaulay. |22l Corollary 
1.6] 

(2) If P is sequentially Cohen-Macaulay, then all intervals in P are also se- 
quentially Cohen-Macaulay. |221 Theorem 1.5] 

Then in the previous two sections we have actually shown 

Proposition 2.12. If €(G) is sequentially Cohen-Macaulay, then G is a comple- 
mented group. 

Proof. The proof of Proposition ^. 4l shows that if P is a p-group, but not elementary 
abelian, then £(P) has the homotopy type of a wedge of spheres of the wrong 
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dimension. Hence the homology does not vanish below the top dimension, and 
<£(P) is not (sequentially) Cohen-Macaulay. Lemma, 12.1 II part 2 then gives that all 
Sylow subgroups of a group G with £(G) sequentially Cohen-Macaulay must be 
elementary abelian. 

Similarly, in the proof of Theorem 12.101 we show that £o(G) is not Cohen- 
Macaulay. By Lemma, l2~8l we have that £o(G) is the pure n-skeleton of £(G), 
and then the definition gives that £(G) is not sequentially Cohen-Macaulay unless 
G is supersolvable. □ 

Proposition 12 . 1 21 and the fact that complemented groups are supersolvable then 
give that (6) and (7) are equivalent, and that both imply (1-3) in our Restatement 
of the Main Theorem. Then (4) =^ (5) =>■ (6) is clear from the definition of 
homotopy Cohen-Macaulay, and it remains only to prove (1-3) =>■ (4). This will 
be the subject of Sectional 

3. Some linear algebra 

We now take a brief break from shellings and homotopy type to do some linear 
algebra. First, we introduce some notation. Fix a vector space V with (ordered) 
basis 25 = {ei, . . . , e„}, and consider a subspace U QV . Let {gi, . . . , gu} be a set 
of generators for U. Then we can write the coordinates of the gi's as row vectors 

9i 

[<?i]o3, put these in a matrix : , and reduce to reduced row echelon form M. 

Denote the set of pivot columns for M (i.e., the columns with a leading 1 in some 
row of M) as Iv,<b(U), or just I{U) if the choice of V and 25 is clear. 

Lemma 3.1. I{U) is an invariant for the subspace UofV with respect to 25. 

Proof. We need only show that I{U) does not depend on the choice of generators 
for U. Suppose generators {hi} give row reduced matrix Mh and generators {gi} 
give row reduced matrix M g . But then the row reduced matrix of {hi} U {gi} must 
be both Mh and M g by uniqueness of reduced row echelon form, hence M g = Mh- 
In particular, the pivot columns are the same. □ 

We mention some elementary properties of our invariant. 

Proposition 3.2. Fix V and 25 as above, and let U\,Ui be subspaces ofV. Then 

(1) |I(E7i)| = dimE/i. 

(2) IfUi C U 2 , then I{U X ) C I(U 2 ). 

Proof. From a first course in linear algebra, the number of pivots of a matrix is the 
dimension of the row space, and adding rows to the matrix adds pivots, but does 
not change the ones we had before. □ 

We will need the following lemma in our application of I{U) to the next section. 
Briefly, part (2) will correspond with having a unique lexicographically first path 
in intervals of £(G) . 

Lemma 3.3. Fix V and 25 as above, and let U\ C U2 be subspaces ofV. Then 

(1) If k is the largest number in I{U2) \ I{U\), then there is a unique subspace 
such that U x C C U 2 and J(VF T ) = 7(Z7 X ) U {k}. 
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(2) If j is the smallest number in I(U 2 )\I(Ui) , then there is a unique subspace 
W t such that U x C W x C U 2 and I(Wi) = I(U 2 ) \ {j}. 

Proof. 1) It is immediate from the definition of I(U 2 ) that there is some g € U% 
with a 1 in the fcth coordinate, and O's in all preceding coordinates when written 
as a vector with respect to 58. Suppose g\ and g 2 both have this property. Then 
gi — g-2 is in all coordinates up to and including k, hence g\ — g 2 € U\. It follows 
that the desired Wf = (Ui,g) is unique. 

2) First, such a subspace exists. Suppose U 2 — (<?i, ■ ■ ■ , g n ), where the g^s 
are row reduced as in the definition of I(U). Reorder so that gi,...,gi are the 
generators (rows) with pivots in I(U 2 )\I(Ui), ordered from least to greatest (where 
I = \I(U 2 )\I(Ui)\). Then U 2 = (Ui,gi, . . .,gi), and W| = (Ui,g 2 , ... ,3/) is a space 
with the desired properties. 

Suppose W is another such space. Represent W = (Ui, h 2 , . . . ,hi) in the same 
way as we did for U 2 in the preceding paragraph. Let Wo = (g 2 , ...,<?;, h 2 , . . . ,hi). 
Then the hi's and gi's are all zero in coordinates up to and including j, so j £ 7 (Wo). 
Also, Wq C U 2 so /(Wo) C 1(1/2). But the gi's and hi's were row reduced with 
respect to Ui, so are zero in all pivots of Ui, so I(U\) H /(Wo) = 0. To summarize, 
I(W ) C I(U 2 ) \ ({j} U/(C/i)). But since W is at least I - 1 dimensional (as 
the g^s are linearly independent), we get that this is actually an equality. Thus, 
(g 2 , ...,gi) = (h 2 , ...,hi) = W , and we have that W = Wj.. □ 

This ends our excursion into linear algebra. We are now ready to apply the 
results of this section. 

4. Shelling the Coset Poset 

To show that the coset poset of a finite complemented group G is shellable, we 
actually exhibit a co-EL-labeling. First, let us recall the definition of an i^L-labeling. 

A cover relation is a pair x <— y in a poset P such that x ^ y and such that there 
is no z with x ^ z % y. In this situation, we say that y covers x. We recall that 
the usual picture one draws of a poset P is the Hasse diagram, where we arrange 
vertices corresponding with the elements of P such that x is below y if x < y, and 
draw an edge between x and y if x <— y. We say that a poset is bounded if it has a 
unique top and bottom, that is, unique upper and lower bounds. 

Let A be a labeling of the cover relations (equivalently, of the edges of the Hasse 
diagram) of P with elements of some poset L - for us, L will always be the integers. 
Then A is an EL-labeling if for every interval [x, y] of P we have i) there is a unique 
(strictly) increasing maximal chain on [x,y], and ii) this maximal chain is first 
among maximal chains on [x, y] with respect to the lexicographic ordering. If A is 
an -EL-labeling of the dual of P, then we say A is a coEL -labeling. 

Bjorner first introduced -EL-labelings in 0, and showed that if a bounded poset 
P has an LL-labeling, then P is shellable. For this reason, posets with an EL- 
labeling (or coi?L-labeling) are often called EL-shellable (or coEL- shellable). As 
we mentioned before, we will use the invariants I(U) discussed in the previous 
section to construct a coi?L-labeling of £(G). 

Let G = Gi x • ■ ■ x G r be the direct product of square free groups {Gi}, and 
identify each group Gi with its inclusion in G. Fix p a prime. Let H be a subgroup 
of G, with H* a Sylow p-subgroup of H. Let G* be a Sylow p-subgroup of G, with 
H* contained in G* . By the normality of Gi and an order argument, we get that 
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G* C\Gi is either isomorphic to Z p or 1 (depending on whether p\ |Gj|). Let e, be a 
generator of G* fl Gi when this intersection is nontrivial. Let 03 be an ordered basis 
of such generators ej, taken from each nontrivial G* PI G,, Think of the elementary 
abelian subgroup G* as a vector space over Z p , and define I P (H) to be Iq* ,<b(H*). 

Lemma 4.1. I P (H) is well- defined. 

Proof. We need to check that I p (H) is independent of the choice of H* , G* , and 
03. Recall that Ig*,<b{H*), as defined in Sectional is the set of pivots of the matrix 
with rows generating H* . 

Notice that an element g — e" 1 • • • e" r £ G* has ctj — for j < i if and only if 
g £ GiGi+i ■ ■ ■ G r . Now the pivot associated with is in Ig*,<b(H*) if and only 
if there is an element in the matrix of row-reduced generators for H* with first 
non-zero position i, that is, of the form h — • • • e" r with a; ^ 0. We see that 
this happens if and only if h £ Gi ■ ■ ■ G r \ G i+1 ■ ■ ■ G r . We also notice that a set 
of generators is (weakly) row-reduced if and only if the first non-zero positions are 
strictly increasing. Thus, the set of pivots for H* is determined by the subgroups 
of the form Gj ■ ■ ■ G r , and as Gj ■ ■ ■ G r does not depend on G* or 03, it follows 
immediately that I P (H) is independent of them. 

It remains to check that I P {H) is independent of the choice of H* . Any alternate 
choice differs only by conjugation by some element x £ H. But a set of generators 
{hi, . . . , hk} for H* has the same matrix representation with respect to 03 as their 
conjugates {x~ 1 hix, . . . , x~ 1 hkx} has with respect to x~ 1( Bx. In particular, the 
set of pivots is unchanged. □ 

We need a couple more lemmas. 

Lemma 4.2. Let M be a maximal subgroup of a supersolvable group G. If G = 
HM , then Hx fl M is a maximal coset of Hx. 

Proof. Since G = HM, we can write Hx — Hra for some m £ M . So Hx fl M — 
(H n M)m j£ 0. Also, \G\ = \HM\ = = [H : H n M] ■ Since [G : M] 

is prime, it follows that [H : H fl M) = [G : M] is also prime, hence that H fl M is 
maximal in H. □ 

The following lemma (due to G. Zappa) is proved, for example, in jl 71 5.4.8]. 

Lemma 4.3. Let G be a finite supersolvable group. Then G has a chief series 
1 = N C ■ • • C N k = G with [JVi : A^o] > [N 2 : N ± ] > ■ ■ ■ > [N k : N k -i]. 

In particular, if p is the largest prime dividing |G| and q is the smallest; then G 
has a normal Sylow p-subgroup and a normal Hall g'-subgroup. 

Corollary 4.4. Let G be a finite supersolvable group. If p is the smallest prime 
dividing [Ho : H n ] for subgroups H n C Hq of G, then there is a unique subgroup 
H\ with H n C Hi C Ho and such that p does not divide rg^rffn ■ 

Proof. Let n = {q : q < p, q \ \H \}, and A' be a Hall 7r'-subgroup of H . Then K< 
H by the lemma, hence KH n is a subgroup of H with the desired properties. □ 

We are now ready to prove the Main Theorem. The high level idea is to use 
the changes in the invariants I P (H) to label cover relations. Unfortunately, that 
gives us a lot of identically labelled chains. So we pick out some distinguished cover 
relations, and change their labels to have a unique increasing chain. The details 
follow. 



SHELLING THE COSET POSET 



11 



Theorem 4.5. If G is supersolvable with all subgroups elementary abelian, then 
C(G) is coEL-shellable, and so G is coset-shellable. 

Proof. We recall by the theorem of P. Hall restated in Theorem ll.5l tha,t G C Gi x 
• ■ ■ x G r where each Gi is of square free order. If £(Gi x • ■ ■ x G r ) is coi?L-shellable, 
then it follows immediately from the definition that the interval [0, G] = £(G) is 
as well. So we can assume without loss of generality that G = G\ X • • • X G r , the 
direct product of groups of square free order. 

For each i, and each p dividing |G,|, pick M pi to be a maximal subgroup of index 
p (a Hall p'-subgroup) in Gi. Such M*^s exist because Gi is solvable, and it is 
a well-known characterization of solvable groups that they have Hall p'-subgroups 
for each prime p (see ^3 Chapter 9] for more background). Then set M p i to be 
x rX/jj Gj- Fix l(p,j) to be an order preserving map into the positive integers 
of the lexicographic ordering on the pairs (p,j) for all p dividing \G\ and j — l,...,r. 
We will use M p j to pick out the distinguished edges mentioned above. Most edges 
will be labeled with l(p,j) for an appropriate p and j, while these distinguished 
edges will be labeled with the negative of l(p,j). 

More precisely, suppose H%x C Hqx is a cover relation in £(G). Since G is 
supersolvable, [Hq : H{\ = p for some prime p, hence the Sylow p-subgroups of Hi 
have dimension (as vector spaces over 1 p ) one lower than those of Hq. It follows 
that I P {H ) = I p {Hx) U {j} for some j. Then label the edge H x -> H x x as 



X(H x -> Hix) = 



—l(p, j) if Hix — H x fl M p j 
KPij) otherwise. 



Finally, label \(x — ► 0) = 0. We will show that A is a co-EL-labeling. 

Intervals in £(G) all have either the form [®,H x], or [H n x, H x}. We consider 
these types of intervals separately, and show there is a unique increasing chain 
which is lexicographically first. 

On [0, Hqx], we notice from Proposition 13 . 21 that every maximal chain from Hqx 
down to has on the last edge, and ±l(p,j) (over all pairs (p, j) such that 
j G I p (Ho)) on the preceding edges. In fact, for each such pair (p,j), exactly 
one of +l(p, j) or —l(p,j) occurs exactly once on any maximal chain. Finally, since 
j e I p (Hq), there is an element of order p in Hq of the form ejBj+i ■ ■ ■ e r , where each 
G Gi and ej ^ 0. As Gi C M p j for i ^ j, we see that HqM p j = HoM p jM Pt j C 

Then since is the last edge, the only possible increasing chain is the one with 
labels —l(p,j) in increasing order. By Lemma l4~2l there is such a chain, it is clearly 
unique and lexicographically first. 

For [H n x,Hox], the situation is only slightly more complicated. Let a pair 
(p,j) be called admissible for the given interval if p divides [Ho : H n ] and j 6 
I P (H ) \ I p (H n ). If l(p,j) is minimal among admissible (p,j), then there is a 
unique H\x of index p in H x with H x — > H\x labeled ±l(p,j) by Corolla.rv 14.41 
and Lemma 13.31 Moreover, any chain on [H n x, Hqx] has exactly one edge with 
label ±l(p,j) for each admissible (p,j). 

Suppose G is an increasing chain on [H n x, Hqx}. Suppose HiX — > Hi + \x in G 
is labelled +l(p,j). Then l(p,j) is minimal among (p,j) admissible for [H n x, H^x] 
since the chain is increasing. Thus Hix — > Hi + \x is the unique edge down from 
HiX labeled with ±l(p,j), and since the label was positive we see that H n x % M p j. 
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It follows that the unique increasing chain on [H n x, H x] is the lexicographically 
first one labeled with —l(p,j) in increasing order for (p,j) such that H n x C M p j, 
followed by +l(p,j) for all other admissible (p,j). □ 

5. Examples 

At first glance, the labeling constructed in Theorem 14,51 might seem to come 
"from left field." It is helpful to work out what happens for the case where G is 
a group of square free order. In this case, many of the complications we faced 
in the proof disappear. For example, we don't have to worry about I p , since if 
[H : H x ] = p, then = and I P (H ) = {1}. Similarly, we can just take 

KPyj) = Pi since the only possible value of j is 1. The only M p j's we have are 
M Pt i, which we can denote as M p . 

Thus we see that for any H , Hi with [Ho : Hi] = p we get 

X(H x -> H ) = { ~ P if HlX = H ° X n Mp 
\ p otherwise 

and X(x — > 0) = 0. An example for is worked out in Figure An exercise for 
the reader might be to work out the labeling for S3. 




Figure 2. The coi?L-labeling of <£(Z 6 ). The leftmost two maxi- 
mal cosets are M3 = and Mi = M2,i, respectively. 



On the opposite extreme, it is not so hard to understand the co-EL-shelling on 
Z™ - it is just the change in I p , with l(p,j) becoming j. We will not say anything 
more about this, but an example for Z| is worked out in Figure El 

6. Consequences and Conclusion 

A (co-)-EL-labeling of a lattice L tells us a lot about the homotopy type of 
L\{0, 1}. In particular, the falling chains (for our purposes, weakly decreasing max- 
imal chains) in an i?L-labeling give a basis for the nontrivial homology /cohomology 
group. See 0, Section 5] for a discussion of this in a more general setting. Our 
co-EL-labeling for £(G) (where G is a complemented group) thus helps us under- 
stand the cohomology of the order complex in a very concrete way. 
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Figure 3. The co-EL-labeling of £(Z§). The leftmost two maxi- 
mal cosets are M2,2 and -M24, respectively. 

In showing the shellability of a solvable group's subgroup lattice, Shareshian 
[19] produces a so-called "coatom ordering." Unfortunately, while the existence 
of a coatom ordering implies the existence of something with similar properties 
to a coi?L-labeling (a "coCL-labeling"), Shareshian is not able to exhibit such a 
labeling. Such a labeling would be interesting, as it could presumably be used to 
give an alternative proof and/or expand upon a result of Thevenaz [21 Theorem 
1.4]. Perhaps techniques like we use here could be used on the chief series for a 
solvable group (where every factor is an elementary abelian p-group) to produce a 
(co-)-EL-labeling in the subgroup lattice. 
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